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ABSTRACT 



Preparing courses for flexible delivery and distance 



education is normally a time-consuming and expensive process. This paper 
describes the design and evaluation of a system that automatically captures 
and indexes audio and video streams of traditional university lectures 
without demanding any changes in the style or tools used by teachers. Using a 
"Wizard-of -Oz " technique to simulate the automatic indexing, a 4 -month trial 
of the system in a large (746 students) first year Computer Studies course 
was run. The results reveal some surprising social implications of making 
flexible delivery available to students at a residential university. Early in 
the. trial, many students expressed an intention to use the system, but few 
used it. Late in the course, many students stated that they urgently needed 
the system for revision, but even fewer used it. At the same time, lecture 
attendance appeared to be lower than normal. It is hypothesized that the 
availability of a flexible alternative to lectures removed the necessity of 
attending lectures, and that students deceived themselves about their 
intentions to catch up using the digital medium. (Contains 18 references.) 
(Author/AEF) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



I 



f 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 

G.H. Marks 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



Flexible Delivery Damaging to Learning? 
Lessons from the Canterbury Digital Lectures Project 



Tim Bell 1 , Andy Cockbum 1 , Bruce McKenzie 1 , John Vargo 2 
1 Department of Computer Science 
2 Department of Accountancy, Finance and Information Systems 
University of Canterbury, Christchurch 
{tim, andy, bruce}@cosc. canterbury. ac.nz 
j.vargo@afis. canterbury. ac.nz 



o«V^ s ;, D p! partment of education 

Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
a , CENTER (ERIC) 

¥ This document has been reproduced as 
/ received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



LT) ! 

m 



VO 

VO 



p 



w 



Abstract: Preparing courses for flexible delivery and distance education is normally a time- 
consuming and expensive process. This paper describes the design and evaluation of a system that 
automatically captures and indexes audio and video streams of traditional university lectures 
without demanding any changes in the style or tools used by teachers. Using a 'wizard-of-oz' 
technique to simulate the automatic indexing, we ran a four-month trial of the system in a large 
(746 students) first year Computer Studies course. The results reveal some surprising social 
implications of making flexible delivery available to students at a residential university. Early in 
the trial, many students expressed an intention to use the system, but few did. Late in the course, 
many students stated that they urgently needed the system for revision, but even fewer used it. At 
the same time, lecture attendance appeared to be lower than normal. We hypothesise that the 
availability of a flexible alternative to lectures removed the necessity of attending lectures, and that 
students deceived themselves about their intentions to catch up using the digital medium. 



Introduction 





‘Flexible delivery’, ‘distance education’ and ‘virtual learning’ have become popular buzzwords within academia. 
There are, however, many technological, financial and social constraints that limit the practicality of the ‘glamorous’ 
visions of virtual learning involving real-time distributed classrooms supported by high-bandwidth multi-party video 
interaction. High-bandwidth networks will eventually be sufficiently ubiquitous to enable rich virtual learning 
environments with equitable participation possibilities for proximate and remote students, but current 
implementations are severely constrained by bandwidth and other considerations. 

In 1999 the leaders of our university — the University of Canterbury, New Zealand — announced their intention 
to equip all lecture theatres on campus (approximately 20) with the necessary equipment to support virtual learning. 
Unsure what this would entail and at what cost, the authors of this paper proposed a trial research project to 
investigate practical methods for flexible delivery of lectures. 

Without massive institutional change, it was clear that Canterbury academics would not have time to develop 
extensive course materials for virtual learning. With this constraint in mind, we focused on designing, implementing 
and evaluating a system that would maximise the pedagogical benefits to students while minimising (to zero if 
possible) the additional work for the academics who deliver and administer the courses. Having designed and 
implemented the system we trailed it as a resource available to the 746 students taking a year-long first year 
‘Computer Studies’ course. 

This paper describes the results of the four-month trial of the Canterbury ‘Digital Lectures’ system. The system 
was designed to automatically capture and index traditional lecture content without requiring course teachers to 
change their presentation styles in any way. Students who were unable to attend lectures at their normal time and 
place would be able to view them by accessing CD-ROMs held at the university library and in computer labs. 

The following section provides the background design rationale for our system by reviewing various possible 
implementations of ‘virtual learning’. We then describe our system and its objectives, followed by a discussion of 
the unexpected findings from the evaluation. The final section concludes the paper. 
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Dimensions of Virtual Learning 

This section presents four dimensions of ‘virtual learning’ that critically influence the capabilities of technology for 
enhancing learning. 

Learning media 

The traditional lecture remains a defining element of most university courses. Although there are many ways to 
stimulate interaction and small-group work within the lecture environment (for example, see Fischer (1998)) lectures 
remain, at least on the surface, a largely one-way interaction from the professor to the students. The course notes, 
readings, and text-book support students in clarifying and elaborating on the ‘seeds’ of knowledge that are 
(hopefully) planted during the lecture. Arguably the most valuable part of traditional courses occurs when students 
actively test, articulate, and manipulate their understanding. This normally occurs in seminars, tutorials, labs, and 
assignments. 

A wide variety of computer-supported media can be used in an attempt to enhance, offer a surrogate for, or 
replace these traditional ‘media’. For example, it is becoming common for students to bring tape-recorders to 
lectures, but the lack of any indexing into the audio-stream severely inhibits the potential utility of the recordings for 
a student who ‘understood all of a lecture except for the bit on topic X*. Systems such as the Classroom 2000 project 
(Abowd 2000), the Cornell Lecture Browser (Mukhopadhyay and Smith 1999), and our Digital Lectures project aim 
to overcome these limitations by automatically capturing and indexing lecture content. 

A key distinguishing characteristic of new computational learning media is the level of activity required from 
students. Passive media simply provide a static resource for the students. Recording and indexing lectures and the 
now standard practice of putting course materials on the web are both examples of relatively passive media. Passive 
approaches to learning media have been criticised as ‘gift-wrapping’ (Fischer 1998), but it seems reasonable to 
expect that they can provide improvements to courses at relatively low costs. For example, web-based course 
handouts can be updated and improved on-demand, and indexed lecture recordings can aid students who were 
unable to attend the original lecture. 

In contrast to passive media, active media provide an interactive resource that students can use to test and build 
their understanding. There are three types of active educational systems. Firstly, interactive simulation and 
exploration environments let students explore the effects of changing properties within an interactive space. For 
example, animations of algorithm execution (see Brown and Sedgewick (1985) for an early example) allow students 
to explore the algorithm’s behaviour by changing data and parameter values and observing the resultant behaviour. 
Secondly, intelligent tutoring systems (Sleeman and Brown 1982) attempt to model the student’s understanding of a 
problem domain, and then tailor the information they present appropriately. Thirdly, reflection and discussion 
spaces, such as bulletin board and Net News systems, allow students to build a repository of information that 
interlinks contributions by a potentially wide range of participants and sources. Examples include the Dynasites 
system (Fischer 2000) which was explicitly constructed as an environment to support lifelong learning, and a wide 
range of design-rationale and FAQ type systems such as the Answer Garden (Ackerman and Malone 1990) and 
gIBIS (Conklin and Begeman 1998). 

We do not advocate either an exclusively passive or active approach to computational media in support of 
learning. Rather we see that the optimal solution will combine and integrate a variety of media to allow users to 
choose tools that best suit their needs at a particular time. 

Time and place of learning 

A key feature of ‘flexible delivery’ through computer supported learning tools is that it frees students from the need 
to be in a particular place at a particular time. 

There are several important implications of this capability. Firstly, the flexibility gained broadens the potential 
student base, opening a wide range of educational possibilities for ‘lifelong learning’ for the full-time employed and 
those with family commitments. Secondly, the perennial problems of time-tabling clashes need not exclude students 
from particular courses. Thirdly, lecture theatre capacity no longer needs to constrain course sizes. Fourthly, on-line 
media can be reviewed as many times as the student feels is necessary. Traditional lectures, in contrast, are a once- 
only medium, and students who missed the lecture or temporarily lost attention have no way to review the real-time 
explanation. Finally, the freedom from time and place constraints opens the possibility of globally competitive 
education markets. Commercial Internet-based education resources such as stanford-oniine.stanford.edu and 
unext . com are early entrants into this market. 
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Support for collaboration 

Traditional courses, taught at residential universities, provide a natural infrastructure for supporting and promoting 
peer learning. One of the risks of ‘distance education’ is that the lack of physical proximity will act as a barrier to 
this valuable resource. 

On-line systems can support collaboration through simple text-based bulletin boards, through collaborative 
simulation and exploration environments that explicitly account for multiple users (for example TurboTurtle’s 
synchronous collaboration (Cockbum and Greenberg 1998) and AgentSheet’s asynchronous collaboration 
(Repenning, Ioannidou, and Phillips 1999)), or by supporting collaboration around a passive medium such as 
recorded lectures. Of particular interest to us is the notion of supporting collaboration around video recordings of 
lectures. In an early study on video-based instruction, Gibbons, Kincheloe and Down (1977) showed that learning 
can suffer when students watch lectures individually. However, by supporting discussion and collaboration around 
the video using the “Tutored Video Instruction” (TVI) model, learning improved over traditional lectures. 
Subsequent research, such as that reported in Cadiz, Balachandran, Sanocki, Gupta, Grudin, and Jancke, (In Press) 
has begun to investigate distributed versions of the TVI model, where physically remote students are connected to 
each other by multiple audio/video feeds. 

A final issue is that of equity between students participating remotely and locally. Anecdotal evidence indicates 
that remote students in ‘live’ video lectures (one-way video, but two way audio) felt like ‘second-rate participants’ . 
Some of the students, however, may value the more relaxed atmosphere of a video-linked lecture — for example, 
walking out of the theatre mid-lecture without disturbing, or being seen by, the lecturer. 

Capital Investment, Effort and Social Factors 

The final dimension that critically affects the nature of on-line learning support is the degree of capital investment 
and effort put into developing the resources. 

Institutions with a long history of supporting distance education — for example Stanford University in the U.S. 
and the Open University in the U.K. — make large capital investments in producing high quality media such as 
broadcast quality TV lectures/documentaries to support their courses. These financial outlays are then amortised 
over thousands of students across the entire nation, and over several years. 

Clearly, not all teaching institutions can compete for distance learning students against highly commercialised 
alternatives. The question then becomes the following: how can traditional residential universities best use their 
available resources to enhance their support for learning? 

Beyond the resources dedicated to on-line courses, there are also many social factors that must be addressed in 
preparing courses for flexible delivery. Systems that depend on lecturers and course administrators changing their 
teaching habits may promise a wide range of learning benefits to students, but if the lecturers fail to make the 
necessary changes then no benefits can be realised. Furthermore, many lecturers have concerns about issues such as 
copyright, liability and job-protection when every action and utterance in the lecture theatre is captured on multiple 
media. 

Canterbury’s Digital Lectures Project 

The goal of Canterbury’s Digital Lectures project is to develop mechanisms for flexible delivery of conventional 
lectures that enhance learning while requiring no modifications to the lecturer’s teaching methods. Like related work 
on the Cornell Lecture Browser our goal is “to allow a speaker to walk into a lecture hall, press a button, and give a 
presentation using blackboard, whiteboards, 35mm slides, overheads, or computer projection. An hour later, a 
structured document based on the presentation will be available [...] for replay on demand” (Mukhopadhyay and 
Smith 1999). 

The main difference between our work and that of the Cornell Lecture Browser is that we focus on the ways in 
which students used the end-products, while Mukhopadhyay and Smith focus on the technology used to generate the 
end-product. 

The Student’s Interface 

Figure 1 shows the prototype interface to a digital lecture. In this prototype there are two video streams, represented 
by the thumbnail images at the top-middle of each window. In Figure 1 the user has chosen to watch a broad video 
stream showing the front of the lecture theatre, including the lecturer and the overhead projection screen. In 
Figure 2, the user has clicked on the other thumbnail image to zoom into the text on the overhead material. 
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Figure 1: The student's interface showing a 
broad view of the theatre. 




Various controls allow the user to freely navigate through the lecture. As well as the normal ‘fast- forward’ and 
‘rewind’ controls, the user can click on the time-line (beneath the control-buttons) to jump to a particular time in the 
lecture. Also, the right-hand side of the interface provides a hypertext keyword index to the lecture. Clicking on an 
item in the list causes the video and audio streams to immediately jump to the associated part of the lecture. 

The actual interface that the students used in the trial, reported in the evaluation section, was identical to this 
interface except that it did not support multiple video-streams. 



Capture and distribution 

In investigating automatic capture techniques, we explored a range of research challenges such as optical character 
recognition from low-resolution video images, and automatic detection of changes in video streams (in order to 
detect changes in the state of the blackboard or overhead slide, etc.) These technological issues are further described 
in Adjeroh, Bell, Cockbum, McKenzie, and Vargo, (2000). 

While continuing to work on methods for automating the creation of digital lectures, we ran a four-month trial 
of the student’s interface to the digital lectures in a large first year ‘Computer Studies’ course. During this trial we 
used a ‘wizard-of-oz’ technique (Gould, Conti, Hovanyecz, 1983) that used human input to simulate the automatic 
capture of the digital lecture. The student’s interface, however, was unaffected by the ‘wizard-of-oz’ data capture. 
During the lecture, a camera-person operated the camera. The feasibility of automated camera management is 
demonstrated by Liu, Rui, Gupta and Cadiz (2000) who show that most people could not tell whether lecture room 
video was captured by a person or by an automatic system. After our lectures, an operator ran a series of programs 
that captured and burnt the lecture contents onto a CD-ROM. The programs translated the digital video (DV) tape 
into a motion-JPEG stream with a synchronised MP3 audio stream. Finally, the operator watched the video and 
created hypertext links — identifying topics in the lecture — synchronised with the video stream. Again, the feasibility 
of automatic techniques is demonstrated by Mukhopadhyay, and Smith (1999) who describe techniques for 
automatically identifying slide changes and by Wellner (1993a, 19936) who discusses techniques for retrieving text 
from video-images. 

The CD-ROMs (one for each lecture throughout the course) were available for students to use in the university 
library. Using CD-ROMs as a distribution medium allowed us to closely monitor use of the medium, and we 
required students to complete a questionnaire each time they used a CD-ROM. 



Evaluation and Results 

In the second semester of the year 2000 academic year (July to October), digital versions of each lecture in the large 
(746 students) first year Computer Studies course were made available to students. At the start of the study, students 
who wanted to use the digital lectures were required to sign an ethical consent form. Ninety- four students requested 
to do so. 

Initial use of the system was extremely light, with only four students attempting to use the system throughout 
July and August. At the same time lecture attendance appeared to be lower than during the previous term. 

In September only two people accessed the digital lectures. As a result, the course lecturer sent an email 
message to the class asking whether it was worthwhile continuing to make digital lectures available. Much to our 
surprise (given the extremely low use of the system until then), fifty-three students replied, strongly urging us to 
continue producing the lectures. Comments from the students in their email replies indicate that many had been 
intending to use the digital medium for some time, but had not got around to doing so: 

> Do you intend to view the Digital Lectures before the COSC110 exam? 

Yes I do intend to use the lecture disks but mainly for the lectures I've missed 
or need to brush up on (thats probably all of them if I was completely honest) 
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> When are you likely to want to view them? 

I intend doing this over the next few weeks when time permits (I work full time) 

> Have you tried to use the Digital Lectures already? 

No I haven't used them at all yet, so I'm one of the guilty that signed up and 
not used them. 

Despite the appeals for us to continue, only seven students subsequently accessed the CD-ROMS. 

Discussion 

It appears that students deceived themselves about their intentions to catch up on lectures using the digital 
alternative. We hypothesise that by making lectures available at any time, we removed the need for students to 
attend the ‘live’ lecture. Normally, the ‘live’ lecture is a one-off event, and if students miss it their chance to benefit 
from it is gone. The damaging part of the digital media (in our experiment), however, is that the students never 
fulfilled their intention to access it. Gerhard Fischer (private communication) suggested a ‘local traveller’ analogy to 
explain this effect. It is not uncommon for residents in an area to have never visited some of the local attractions: 
they intend to, but the attractions are constantly available, and there is no pressing need to visit them immediately. 
Tourists visiting the area, however, typically have only one opportunity to see the sights, and consequently make 
concerted efforts to ensure that they exploit it. By making lectures freely available to the students they were able to 
postpone viewing the lectures because “they’ll still be available tomorrow”. Like local residents, these intentions 
are sometimes postponed indefinitely, or until too late. 

Conclusions 

Advances in desktop multi-media and the World Wide Web have brought about a wide range of new possibilities for 
education. Many research projects are exploring the possibilities of the technology while others are deploying first 
generation systems. 

This paper investigated some of the possibilities and problems of virtual learning systems. In particular, we 
reported the somewhat surprising results of a four-month trial of the Canterbury Digital Lectures Project. In essence, 
the results of our trial suggest that providing too much flexibility to students at a residential university can have 
negative learning results. By providing a digital surrogate for ‘live’ lectures, it appears that students made less effort 
to attend the lectures, and that they never fulfilled their intention to catch up using the digital media. 

In retrospect, it is unsurprising that misplaced technological support can have negative effects on learning (or 
at least on class participation). In our further work we intend to address the questions that identify the conditions 
under which learning technology is well placed. For instance, we would like to know whether our technology would 
have had a positive effect if we had used the web to bring the lectures to students who could not otherwise have 
participated in classes. 
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